Technologies Supporting Effective Air Battle Management

Battle management command and control (BMC2) is a communicationsintensive activity. Weapons directors and mission crew commanders on these platforms are required to monitor as many as eight simultaneous communications channels against a background of moderate to high ambient cabin noise while performing a number of visual and manual tasks, a combination which in the heat of battle is challenging even for the most highly trained operators. Researchers at the Air Force Research Laboratory’s Human Effectiveness Directorate (AFRL/HE) have been investigating two technologies to ameliorate this problem: active noise reduction (ANR) headsets and spatial intercoms. ANR headsets cancel environmental noise while preserving speech signals presented via the communications network, while spatial intercoms enhance communications by increasing the effective signal-to-noise ratio of each individual communications channel, taking advantage of the listener’s natural ability to efficiently segregate speech streams that are separated in space. This paper will describe investigations at AFRL/HE directed at the enhancement of speech intelligibility in multi-channel tactical BMC2 environments using ANR and spatial intercom technology. Results will be discussed from basic laboratory experimentation, simulated mission scenarios, and field evaluations, and interpreted in the context of the acquisition and integration of such technologies for current and future BMC2 weapons systems. Introduction. Battle management command and control (BMC2) as performed by operators of tactical air battle management platforms such as the US Air Force’s E-3 Airborne Warning and Control System (AWACS) or E-8 Joint Surveillance Target Attack Radar System (JSTARS), or the US Navy’s E-2C Hawkeye, is a communicationsintensive activity. Weapons directors and mission crew commanders on these platforms are required to monitor as many as eight simultaneous communications channels against a background of moderate to high ambient cabin noise while performing a number of visual and manual tasks, the combination of which in the heat of battle is challenging even for the most highly trained operators. Researchers at the Air Force Research Laboratory’s Human Effectiveness Directorate have been investigating two technologies that may ameliorate this problem: active noise reduction headsets and spatial intercoms. Active noise reduction (ANR) headsets make use of the phenomenon of destructive interference to cancel low-frequency environmental noise while preserving the speech signals presented to operators via the communications network. This is accomplished by measuring the external noise field at each ear, inverting the phase of the waveform, and adding it to the original waveform by means of a speaker placed inside the ear cup. In addition to improving communications effectiveness, ANR can reduce the hearing loss attendant from long-duration sorties and the practice of increasing the volume of a particular net or radio channel to a dangerous level in order that it might be audible above the compound background of noise and other speech. Spatial intercoms (Bolia & Nelson, 2003) endeavor to enhance communications by increasing the effective signal-to-noise ratio of any particular communications channel, taking advantage of the human listener’s natural ability to more efficiently segregate speech streams that are separated in space. This is accomplished by digitally filtering discrete channels using transformations derived from the physical cues used for sound localization to make the speech on each channel appear to emanate from distinct locations in space. Research has demonstrated that such apparent separation can lead to substantial improvements in speech intelligibility and reductions in communications workload. This paper will describe investigations at the Air Force Research Laboratory directed at the enhancement of speech intelligibility in multi-channel tactical BMC2 environments using ANR and spatial intercom technology. Results will be discussed from basic laboratory experimentation, simulated mission scenarios, and field evaluations, and interpreted in the context of the acquisition and integration of such technologies for current and future BMC2 weapons systems. Active Noise Reduction Headsets. The moderate to high ambient cabin noise present on airborne air battle management platforms such as AWACS and JSTARS and the duration of missions flown point to the potential for hearing loss in both the flight and the mission crews (Mobley, Hall, & Yeager, 2002). This danger is exacerbated by the tendency of some crewmembers to wear their headsets with one ear exposed in order to facilitate communication with other crewmembers without squandering the onboard intercoms. This activity may lead to asymmetric hearing loss, further impairing communication effectiveness. In order to improve communication effectiveness and reduce the potential for hearing loss, the AWACS System Program Office (SPO) decided to purchase ANR headsets to replace the current headset in use by the AWACS crews. Inasmuch as several commercial producers could provide ANR headsets, the AWACS SPO requested that AFRL conduct an evaluation to determine the best overall choice for use in the AWACS environment (Vidulich, Donnelly, McLaughlin, Read, Hall, Schley, Mobley, Bolia, Nelson, & Poole, 2003). The evaluation included several objectively measured headset attributes as well as subjective ratings. The list of objective criteria included: Noise Attenuation Performance, Headset Clamp Strength, Headset Weight, Battery Pack Weight, and the Type and Number of Batteries Required. The values for all of these were determined by measurement or observation in AFRL laboratories. For the purposes of comparison the actual physical measurements on the various objective parameters were not used in the later calculations. For any measurement class, the raw values were converted to a relative value scale that identified how each headset compared to the “average” headset. The subjective evaluation criteria were any opinion-based values derived from the ratings of the operators that tested the headsets in representative flight operations. The evaluation team defined five subjective dimensions to be assessed by the ANR headset evaluators: Fit, Comfort, Usability, Audio Quality, and Hearing Protection & Fatigue. The subjective ratings were collected from AWACS aircrews performing normal aircrew duties during several combat training sorties flown out of Tinker Air Force Base, Oklahoma. The fact that different headsets were assessed by different teams of operators was problematic due to the potential influence of individual differences on subjective ratings. To minimize these influences as much as possible, a relative comparison approach was used (Tsang & Vidulich, 1994). As all of the operators had considerable experience with the current headset, it was used as the baseline and all subjective evaluations were based on comparison to it. This approach had the added advantage of directly testing whether the headsets under evaluation constitute an improvement over the current headset (Rueb, Vidulich, & Hassoun, 1994). A decision matrix procedure was used to combine the raw data according to the relative importance weights provided by subject-matter experts (Halpern, 2002). Overall, the results strongly supported the potential value of ANR headsets in the in-flight environment. ANR headsets were assessed as improving communications throughput, in addition to providing greater hearing protection. Based on the parameters considered in the evaluation, the Bose AHX headset was the clear winner over the other tested alternatives. The subjective evaluations for the Bose headset were strongly positive, the headset weight and head clamp strength measurements were very good, and there were no negative factors found. The Bose headset was also perceived as having better than average attenuation performance among the headsets evaluated. As an extra precaution to be sure that the Bose headset was the best choice, a second evaluation was conducted (McLaughlin, Vidulich, Donnelly, Bolia, & Nelson, 2003). In this evaluation the Bose headset and the second-place headset from the first evaluation were subjected to a longer in-flight usability evaluation. A survey assessment tool was used to collect data and opinions from AWACS crew members as they used the headsets for about four weeks. Overall, the pattern of results strongly supported the Bose headset as a better choice for AWACS use. The ratings and qualitative comments showed that the Bose headset was considered a substantial improvement relative to the current headset, but that the second-place headset was more prone to comfort and usability issues. Based on these two evaluations, the Bose headset was purchased for use on the AWACS aircraft. Spatial Intercoms. That spatial intercoms are possible is due to a phenomenon known as the “cocktail party effect” (Cherry, 1953), which refers to the fact that, at a cocktail party, in the midst of perhaps scores of people engaged in dozens of conversations, an individual can still make sense of what the person she is talking to is trying to say, despite the fact that if the same conversations were played to the individual over mono headphones she would be hopelessly lost. While many factors contribute to the cocktail party effect (see Bronkhorst, 2000; and Yost, 1997, for reviews), one of the most often studied is the spatial separation of competing talkers. It has long been known that listening to two or more speakers simultaneously is easier if the two are at different locations. In the real world this may not be obvious, since multiple talkers seldom inhabit exactly the same spot. However, listening to even two overlapping talkers over headphones, or from the same loudspeaker, is a challenging and often frustrating task. If spatial separation enhances the intelligibility of multi-talker conversations, a display that is able to effectively simulate distinct localizable sound sources might also be adapted for use as a spatial intercom. The construction of such a display is possible because of what scientists have learned about the cues subserving sound localization in humans, all of which are based on the interactions of the incident acoustic wave with the head, torso, and pinnae. Two of these cues, interaural time differences (ITDs) and interaural level differences (ILDs) are used primarily for the localization of the sound source in azimuth. ITDs arise because, for sound sources not in line with a listener’s median plane, the acoustic wave will reach one ear before it reaches the other. Although these temporal differences are small – on the order of tens of microseconds – they are readily detectable by brainstem structures. However, because of limitations in neural phase locking, they are only useful for frequency components below about 1500 Hz (Irvine, 1992). For frequencies above 3000 Hz (a region in which the wavelength of an acoustic stimulus is small relative to the size a listener’s head), the head casts an acoustic shadow such that the stimulus level at the contralateral ear (i.e., the ear on the side of the head opposite the sound source) is attenuated relative to the stimulus level at the ipsilateral ear. The resulting ILDs are useful for localizing sounds in this frequency region (Feddersen, Sandel, Teas, & Jeffress, 1957; Blauert, 1974). For frequencies between 1500 and 3000 Hz, little spatial information is available, and hence such sounds are less localizable by humans. Fortunately, most naturally occurring sounds are broadband and include frequency components in the regions in which both ITDs and ILDs operate. These binaural cues are useful primarily for determining the azimuthal component of sounds. However, a particular set of binaural cues does not uniquely specify a point in azimuth. Rather, since the head is roughly spherical, a sound in the front hemifield would result in an ITD and an ILD identical to those of a sound in the rear hemifield (i.e., 10° has the same ITD and ILD as 170°, in a right-handed coordinate system with 0° as the position directly in front of the listener). Failure by the auditory system to disambiguate the location of such sounds can engender front-back confusions (Gardner and Gardner, 1973; Oldfield and Parker, 1984a;1984b). Both localization in elevation (Roffler & Butler, 1968) and the arbitration of front-back confusions (Musicant & Butler, 1984) are mediated by frequency-dependent level differences occasioned by the interaction of the incident sound wave with the pinnae, and, to a lesser extent, the head and torso. Unlike ITDs and ILDs, these differences are highly variable from listener to listener, and the spatial resolution at which they can be discriminated is much poorer than that of either binaural cue. The spectral and temporal differences described above can be measured on human listeners and captured in what is called the head-related transfer function (HRTF; Wightman & Kistler, 1989). If HRTFs are measured for both ears for broadband sound sources at a number of spatial positions, each transfer function can be represented as a digital filter, the collection of which produce a model of the listener’s auditory world that can be used to generate veridical spatial perception. Specifically, given a dense enough set of HRTFs, an arbitrary audio signal can be convolved with the pair representing the transformation from the free field to the left and right ear at a specified position, and be played back over headphones to convey the appearance of emanating from that location in space. Martin, McAnally, and Senova (2001) have demonstrated that such a display can produce localization of “virtual” sounds equivalent to free-field localization. It is well established that headphone-based multi-talker communications can be substantially improved by separating talkers in a spatial audio display. Studies conducted at the Air Force Research Laboratory (e.g., Ericson, Brungart, and Simpson, 2004; Nelson, Bolia, Ericson, & McKinley, 1998) indicate that improvements in intelligibility may be in excess of 30-40%, depending on the configuration and number of simultaneous talkers. However, little is known about how best to position these talkers in order to support optimal performance. Previous studies have typically operated under the assumption that a maximum separation between talkers leads to best performance. As a result, most displays have employed configurations in which all talkers are equally spaced along the plane bisecting the listener’s ears (the horizontal plane), equidistant from the center of the listener’s head, in the listener’s front hemifield. While this does provide maximum separation between all talkers, it does not take into account the fact that auditory spatial acuity is not homogeneous across the plane (Mills, 1958). Specifically, acuity is best for sources presented directly in front of a listener, and becomes progressively worse as the source is moved toward the periphery. Based on this, and recent findings suggesting that listeners can use differences in distance as a means of segregating multiple talkers (Brungart & Simpson, 2002), researchers at AFRL have proposed a new display configuration, capable of accommodating seven talkers (Brungart & Simpson, 2003). In this display, the talkers are more closely spaced in the front, and more widely toward the periphery. Moreover, both far-field (1 m) and near-field (12 cm) locations were employed. This configuration can be seen in the top panel on the lefthand side of Figure 1. The corresponding performance benefits for this spatial configuration are shown in the right-hand panel of the same figure. These data, collected in the laboratory at AFRL, are compared to performance in a diotic configuration, in which all channels were mixed and presented identically to both ears, as shown in the bottom panel on the left-hand side of Figure 1. The diotic condition is analogous to the intercom system that currently exists in AWACS and JSTARS airborne platforms. As can be seen, performance with the optimal 3D audio configuration is substantially better than the diotic configuration, independent of the number of talkers. Figure 1: Comparison of overall performance with spatial audio and diotic audio displays (shown on the left of the figure) in a multi-talker listening task with all male talkers. The results (shown on the right) have been plotted as a function of the number of talkers in the trial. Error bars represent the 95% confidence intervals for each data point. The clear communication effectiveness advantage found with spatial intercoms in the laboratory suggested the need to introduce this technology into the operational community in order to examine its potential benefit for individuals well-trained to monitor multiple communication channels in complex environments. Therefore, researchers from AFRL brought these displays to operators in the BMC2 community. The first tests of this kind were conducted with members of the AWACS operational community at Tinker Air Force Base (Bolia, 2003). Here, laboratory tasks conducted at AFRL were replicated with trained operators in a controlled environment, and later onboard the aircraft on an AWACS training mission. The results were nearly identical to those found in the laboratory, with operators performing 30-40% better on the communication task with spatial intercoms than with the standard diotic communication system. Similar results were found for members of the JSTARS community at Robins Air Force Base, Georgia, and for the ground-based C2 community at the USAF Weapons School at Nellis Air Force Base, Nevada. Perceived workload was also evaluated for AWACS operators using the NASA Task Load Index (TLX). The results indicated that along all TLX subscales (mental demand, physical demand, and temporal demand, performance, frustration, and effort), workload was found to be lower with spatial audio than with standard diotic displays (Bolia, 2003). All of the studies mentioned above measured the affects of spatial audio on speech intelligibility in relatively context-free environments. A logical next step was the evaluation of spatial intercoms in a simulated mission environment. With this in mind, air weapons officers from Tinker Air Force Base were brought to AFRL to control simulated battlefield air interdiction missions with and without spatial comm. Results indicated that speech intelligibility degraded significantly during the most difficult mission segments, and that spatial audio alleviated the degree of this degradation. While perceived mental workload was relatively unaffected by the intercom manipulation, operators indicated on post-hoc questionnaires that they believed spatial audio would be valuable for improving communications effectiveness (Nelson & Bolia, 2003). On the basis of these field and laboratory studies verifying the operational utility of spatial audio for air battle management, AFRL was able to obtain funding to implement the technology in two ground-based controller stations as part of a communications system upgrade at the USAF Weapons School. This system, which allows users to allocate incoming voice communications to seven different apparent locations, has made it possible for dozens of operators to experience the advantages of spatial audio during the conduct of realistic training missions. Operator feedback about this system has been overwhelmingly positive: “With the use of the volume controls and the various bucket positions I had total SA on the fight. I could tell the difference between my aircraft and the adversaries and was fully aware when the pilots were talking on the common safety frequency. This is the first time in my 16 year career that I have fully heard and comprehended a multifrequency mission.” “Increases SA as to what radio (frequency) someone is talking on. Cuts down on guessing what radio a TX came across.” “I think the system/capability is very useful for certain missions. As an SD listening to 5 or 6 different frequencies, it makes a big difference helping you allocate SA to listen to a particular freq.” These comments indicate not only overwhelming acceptance by the user community, but also the increase in mission effectiveness that is possible due to the enhanced situation awareness (SA) supported by spatial audio (see also Simpson, Bolia, & Draper, 2003; and Simpson, Bolia, & Brungart, in press). Operators were able to maintain an awareness of the multiple simultaneous communication channels that were presented to them, and could determine who was saying what and on what channel. It is clear from these results and comments that having a spatial audio display in an operational environment can provide unambiguous and effective communication, and a clear advantage in the operational battlespace. One concern with introducing spatial intercoms into airborne BMC2 platforms is that the binaural cues used to segregate speech channels might be disrupted by the tendency of personnel to operate with one ear open. In order to test this hypothesis, researchers at AFRL conducted an experiment looking at speech intelligibility as a function of the number and configuration of simultaneous talkers under both earcup conditions (both ears covered, one ear uncovered). The results of this study were surprising, demonstrating that although performance with a spatial intercom was reduced when one ear was occluded, it was never reduced below the level of the current diotic communications system. Moreover, there were some configurations for which performance using the spatial configuration was significantly enhanced relative to the non-spatial control, even in the absence of binaural cues (Bolia, 2002). Thus, although such an earcup configuration is not recommended due to its potential for inducing asymmetric hearing loss, it should not be used as an argument against the introduction of spatial audio technology. Spatial Intercoms and ANR. Given the capacity of both spatial audio and ANR to enhance communication effectiveness, it makes sense to inquire as to whether there might be a facilitative interaction between the two. This would be of particular interest, given that effects of ANR on speech intelligibility have seldom been demonstrated (but see Pellieux, Sarafian, & Reynaud, 1997; Simpson & King, 1997). This may be associated with the fact that ANR is most effective at attenuating noises with frequency components below 1000 Hz, a region which is not particularly important for syllable discrimination, and hence not likely to be critical for message intelligibility (French & Steinberg, 1947). Another possibility is that ANR will actually reduce the effects of spatial audio, since it involves modification of the sound wave before it reaches the listener’s eardrum. In order to explore the effects of ANR on spatial audio, a speech intelligibility evaluation similar to the field investigations mentioned above was conducted aboard an AWACS aircraft during a training mission (Bolia, 2003). In this study, personnel listened to two simultaneous talkers presented, in the spatial intercom condition, on both sides of their heads. Although no overall effect of ANR was discovered, it was found to improve intelligibility in the most challenging noise condition, when the target talker on the right side of the head was masked by highly directional aircraft noise. In this case, ANR significantly improved speech intelligibility. One possible explanation for this phenomenon is that ANR reduced the ambient noise level – and hence enhanced the signal-to-noise ratio – in the region in which ITDs operate, thus allowing the operator to use spatial cues not otherwise available in the noise environment. This hypothesis, however, has not endured subsequent investigation, and an adequate explanation has yet to be found. While these results have not been fully explained, they do raise an interesting possibility. If the noise field in which operators are located is known or can be measured, it should be possible to design an adaptive spatial intercom, so that the locations of talkers change in response to operator location on the platform or changes in the noise environment (Bolia, Nelson, & Morley, 2002; Haas, Nelson, Repperger, Bolia, & Zacharias, 2001). While this presents an intriguing possibility, designers should take care that real-time intercom adaptation not result in loss of situation awareness or reduced intelligibility due to the cost associated with overt shifts of spatial attention. Future Directions. The research findings presented thus far provide compelling empirical evidence of the benefits of advanced interface technologies to enhance communication effectiveness, reduce operator workload, and support the establishment, adaptation, and maintenance of situation awareness. As noted above, these conclusions are the result of a converging methodological approach involving highly-controlled laboratory research, operator-in-the-loop simulation experiments, field observations and evaluations, and subjective feedback from subject matter experts. Accordingly, these interface technologies should be considered in the overall system design, especially in noisy and communication-intense applications domains such as air battle management. It is important to point out, however, that this review is somewhat limited in that its primary focus has been on the effects of interface technology on verbal communication. Although verbal communication will continue to be essential in BMC2 application domains, the networked, massively data-linked architecture will demand additional modes of communication and collaboration. Collectively, interfaces technologies to support these modes are known as collaborative interface technologies, and comprise tools that enable synchronous and asynchronous forms of communication. Current capabilities include email, instant messaging and chat, tele-conferencing, video conferencing, file and application sharing, collaborative workspaces, data mining and autonomous agents, decision support and automation, content, knowledge, and workflow management systems, and interactive data visualization technologies. One the primary near-term challenges will be to understand the capacity of these technologies to provide heightened shared situation awareness and real-time collaboration in network-centric coalition operations. Along these lines, some preliminary work has been conducted by Nelson, Bolia, Vidulich, and Langhorne (2004), which characterized some of the barriers to effective communication among coalition operators. However, a survey conducted by these researchers also indicated that the perceived utility of collaborative technologies to support information sharing and collaboration was substantial. Accordingly, research programs systematically addressing the evaluation of the technologies are warranted.